Fruit Grading System for Feature Extraction and Quality Evaluation by Image Processing Technique

Abstract

Since no deep learning fruit grading system based on digital photos has been devised, there hasn\'t been much progress made in determining whether or not fruit is fresh. The goal of this study is to develop methods for decreasing wasted fruit. The agriculture industry in India is crucial to the nation\'s economy. High quality fruit is in high demand, making fruit grading an important aspect of the agricultural industry. However, grading fruit by hand requires a lot of time and effort and is prone to mistakes. The time you spend with automatic grading is well spent. On both domestic and foreign markets, you may find a plethora of fruit. Due to their delicate nature, fruits need extra care throughout the grading process. Automation is much sought after in agricultural research with the aim of evaluating crop quality. If a country\'s fruits, vegetables, and harvests are of higher quality, it may see an increase in output and GDP. Price points for various types of produce are possible. Rapid and automated quality checks are in great demand in the export sector. The visual attractiveness of fruits and vegetables has a significant impact on their marketability, demand, and supply. Even though this kind of classification might provide very unpredictable results, the vast majority of classification tasks in India are still performed manually. Time is of the essence in this process. Taking the environment into account increases costs and introduces an element of uncertainty. This necessitates the employment of automated machinery for fruit sorting. In this research, we propose a method for the automated classification and analysis of digital fruit photographs. This system of grouping fruits makes it easy to compare their relative merits. The size, colour, texture, shape, and lack of flaws or illnesses all contribute to a fruit\'s overall quality. Quality assurance is an iterative process that occurs throughout the development of both hardware and software. Snap a photo of a tomato, and the computer will automatically sort it into the correct category.

Introduction

I. INTRODUCTION

India and other emerging nations rely heavily on the agricultural sector. Every year, governments around the world invest a sizable amount of money into implementing innovative technology, conducting cutting-edge research on the most effective agricultural practises, and developing innovative solutions to the devastating effects of pests, natural catastrophes, and drought [2]. People in India often inspect the quality of their produce. Sorting by eye is not only laborious and time-consuming, but it also introduces the possibility of human error. Rapid and high-precision machine vision technology has made it feasible to automate the grading process, which should increase productivity and accuracy while cutting labour costs. There has been a recent uptick in study into the feasibility of using technology to sort and grade agricultural products. Fruit cultivated on farms may be given a quality grade based on a number of factors, such as size, shape, colour, texture, and the presence or absence of deformities and illnesses. Manually inspecting fruit before shipping is not a feasible or efficient practise. Grading and sorting by eye is labour intensive, hence it is often done by hand.

Many experts in the field, as a result, reach conclusions that aren't grounded in reality and are thus erroneous. Automated sorting and grading systems may one day replace human fruit sorters [1] due to their many advantages, including higher accuracy and faster processing times. As a result of research into and development of key machine vision models for pear quality detection and sorting operations [2, 3], many computer vision-based systems have been developed for food-color-based agricultural grading applications, allowing for faster deployment of novel approaches to assessing agricultural product quality. The quality of fruit may be evaluated in real time using colour mapping [3].

Machine vision and other image processing techniques are finding increasing use in the fruit industry, particularly for quality inspection and defect sorting. Farmers assess their fruit for quality before shipping it to distant wholesale marketplaces. Fruits are often cited as a wholesome addition to a diet because of their low calorie count and high nutrient density. India's fruit industry uses 535,000 acres of land each year, producing 9.362 million tonnes of fruit [3]. Increasing agricultural exports is a major focus of trade policy. The skin of the fruit is so perishable that it must be handled carefully throughout the grading process.

The overall quality of the fruit is affected by factors such as its size, colour, shape, and amount of skin defects. Ripeness and imperfections are the two most important determinants of fruit quality. Evaluation of fruit is a time-consuming yet crucial process. Automatic grading using computers is being considered as a solution since it would lessen the workload for teachers. Some research suggests that fruit rotting is the result of a chain of biological processes that affect the fruit's structure and chemistry (and hence its nutritional value).

The quality of fruit may be judged visually or by some other method. Scent, chemistry, and overall impression are given higher weight in non-visual grading systems. The natural deterioration of fruit causes its colour to change due to a series of chemical reactions. Humans' innate acute sensitivity helps them to recognise rotten produce. It's often believed that consumption over a certain level has positive effects.

It's a tool for figuring out whether the fruit on offer is safe for human consumption [1]. The findings suggest that bacteria may have a causal role in fruit rot. Aerobic psychoactive Gram-negative bacteria that secrete extracellular hydrolytic enzymes that damage plant cell walls are connected to this cycle, as are heterofermentative lactobacilli, spore-forming bacteria, yeasts, and moulds. Fruits are the result of the breakdown of a structural acidic heteropolysaccharide composed mostly of galacturonic acid, which is produced by biochemical processes in the cell walls of terrestrial plants. Glycolysis is the energy-generating process that occurs when polymeric carbohydrates like starch/amylum and sugar undergo anaerobic degradation [2].

Lesions are often thought to be the result of colonisation and microbial spread, which is the major cause of postharvest fruit rotting [3]. Since a shortage of calcium [4] is what triggers apple cork spots, a vitamin deficit might also lead to their appearance. When polyphenol oxidase (PPO) is exposed to oxygen, it initiates a chain reaction that degrades proteins, pigments, fatty acids, and lipids, leading to a loss of colour and a worsening of flavour and odour [5].

II. LITERATURE REVIEW

Numerous studies assessing the quality of fruits and vegetables for commercial purposes have been done over the last few decades. Apples [4, 5], dates [5, 6], mangos [6, 7], citrus fruits [8, 9], and pears [10] are just some of the fruits that have found extensive usage of machine vision-based systems for applications requiring visual assessment. Examples of machine vision systems designed for use in manufacturing settings include [8, 9] an intelligent system for packaging 2-D irregular shapes, [10] an automated system for planning and optimising lumber production using machine vision and computer tomography, and [11] flexible online visual inspections.

Several physical and chemical variables influence tomato quality. Ripeness of tomatoes may be determined by their colour and firmness [11]. Tomatoes may be classified according to their ripeness using any number of scales and colour charts, however the colorimeter test may be unreliable owing to equipment unreliability at varying ripeness stages [12] and the lack of colour uniformity throughout the tomato. A tomato classifier with an accuracy of 84% was developed by Clement et al. [13] using variables such as colour, size, and weight.

However, dimensional analysis is crucial to the success of this method. Tomatoes are at their best when they are fully mature and free of defects. The colour of tomatoes may be used in automated defect detection approaches [14, 15]. Since colour alone is insufficient for flaw detection, the main problem is that the approaches were only 90% accurate. This study also had a very small sample size.

This complicates the task of classifiers like neural networks in assigning meaningful labels to the hidden information. Multiple methods exist for evaluating tomato quality using MRI and spectroscopic images [16, 17]. These imaging techniques, which provide clear images, help doctors make accurate diagnoses. Obtaining a picture, however, may be time-consuming and costly if many imaging techniques are used.

In [4], the authors propose an algorithm for evaluating the quality of agricultural products. Using state-of-the-art technical methods and image-processing algorithms, the system carries out delivery and quality checks.

In this work [5], the researchers use image processing to track the health of fruit from planting to picking. It's possible that a neural network model may be used to identify rotten fruit. Dates are used to indicate a grading system for fruits [6].

In order to distinguish apples from potatoes, authors [7] recommended employing colour information processing. Computer vision has been proposed [8] as a viable method for classifying bell peppers. Two examples of computer vision systems that have been used for automation purposes are the intelligent system for packaging two-dimensional irregular shapes [9] and the camera image contrast enhancement for surveillance and inspection activities [13]. Textures with intricate patterns [14] are also examined using the microscope.

There have been reports of closed-loop vision systems being used for online process control in industry [15]. A computer vision-based system was proposed as a means of evaluating apple fruit quality. Strawberries are graded on their size, their form, and their colour [16]. It's been said [17] that Apple's employment procedures make use of fuzzy logic.

In [18], the authors offer a fuzzy logic rule-based classification method for diagnosing fruit illnesses. Mrunmayee et al. [19] used neural networks to depict illnesses that might affect pomegranate plants. They recognised and labelled a variety of illnesses. It was proposed by Pranjali et al. [20] that grape vines may be used to identify diseases. They contemplated looking at grape leaves over the course of their investigation. They looked into the possibility that bacteria, viruses, or fungi were to blame for the sickness.

Using an edge detection method, Revathi et al. [21] classify cotton leaf diseases. The HPCCD method was recommended for use in disease diagnosis after a literature analysis of cotton fungal diseases. Using a technique called circulatory threshold segmentation, Jun et al. [22] developed a mechanical method for detecting disease-related flaws in oranges' skin.

1. I. Table of different techniques and their efficiency

1) Step1: Data Collection

In order to analyse outcomes and provide answers to relevant inquiries, it is important to collect and analyse data on certain parameters within a predetermined framework. This article details the methodology we used to categorise fruits according to their apparent level of freshness. There is no publicly accessible data on fruits since this is such a novel area of study; we will need to collect this information ourselves. Here, we show how we gather data on fruits and provide evidence that it faithfully reflects the fruits' relative ripeness.

2) Step2:  Creating Datasets of different fruits

Datasets Due to the dataset's diversity, several of the images of apples, bananas, dragon fruits, oranges, pears, and Kiwi fruits exhibit off-putting lighting or distracting background noise. First, the connection between apparent and real ripeness of fruit is investigated. Carotenoids and chlorophyll, present in small levels in young apple skin [30], degrade and alter the peel's reflectance as the apple ages. When bananas are ripe, they become yellow from a pigment called carotenoids [31]. Pectin, cellulose, and hemicellulose are what's left when water (which accounts for 60-90% of an orange's weight) is removed [32, 33]. Carotenoids and flavonoids are responsible for much of an orange's red hue. The beta-cyanins and betaxanthins in dragon fruit are responsible for its vibrant colour [34]. Kiwi fruit is green because of chlorophylls [35], but when it deteriorates, it becomes an olive brown colour from pheophytins and pyro pheophytins. The green or yellow hue of a pear comes from the high chlorophyll concentration of its skin. Dark blue to black pheophytins and pyrophytins are the byproducts of chlorophyll breakdown [35].

3) Step3: Fruit Handling System / Hardware

The University of Agricultural Sciences in Bangalore generously donated 520 unsorted tomato images for the research. Tomatoes in the dataset span the spectrum from perfectly ripe to completely unripe. The categorization would not have been possible without input from on-the-ground specialists. Hardware for the fruit grading system was built in accordance with Figure 3. The tomatoes in Figure 4 were photographed at a resolution of 512 pixels along the longest dimension by the camera perched above the sorting machinery. MATLAB was utilised to build the image processing capabilities.

In response to a signal from MATLAB, the microcontroller advances the conveyor belt carrying the tomatoes. The tomato has to be positioned precisely so for the greatest photo. In the image processing portion, the captured image undergoes a number of processes, including segmentation, feature extraction, and classification. The system's GUI was developed with an emphasis on ease of use. After taking the picture, it is uploaded to a computer monitor. A checkbox is provided for indicating whether or not the fruit is flawed. If that's the case, the damaged tomato will be sent to the conveyor's special area for rejects. The user may initiate the computer's ripeness check with the click of a button. The fruit on the conveyor belt is separated into containers according to its ripeness. Automatic bins are filled with high-quality tomatoes.

4) Step4:  Capturing image from different angles of a fruit on the conveyor belt.

Conveyor belts come with a wide variety of characteristics that are tailored to the specific technique and product being handled. Both PVC and PU-coated vegetable and fruit belts are commercially available. There are belts that may be left on while being washed. Belts' finishes may range from moulding to sealing the edges. Inclined conveying allows for the coupling of a wide variety of belt surface types (including sidewalls, wave profiles, long or short finger profiles, horizontal or longitudinal profiles).

5) Step 5: Obtaining the Image

In order to process a picture, one must first get it from some kind of source, often some kind of hardware. Since no work can be done on the system until an image is submitted, here is where the process begins. The final product looks almost identical to the source material.

Conclusion

In this research, we show how to use computer vision techniques for automated tomato quality assessment. The hardware of the fruit grading system may then utilise the results from the image processing module to sort the fruit into the proper bins. All shots were correctly labelled as \"defective\" or \"non-defective\" by the algorithm, while 96.47 percent of fruit images were correctly labelled as \"ripe\" or \"unripe.\" Growers and food processors may now effectively sort fruit thanks to a trustworthy system. Since the grading machine can only score 300 fruits per hour, high specular reflection in the fruit photos poses a challenge for the proposed image processing method. Since human grading takes so much time and may lead to contradicting conclusions when undertaken by different persons, there is an urgent demand for fruit grading using machine learning and computer vision. The training dataset cannot be created without first collecting example photographs with a wide variety of flaws. The proposed method may help make automated fruit grading systems more reliable in the long run. Before it can be used in the field, its speed and accuracy must be enhanced.

References

[1] Sohoo S. ICT initiative of SAARC agriculture centre in the SAARC region, International Conference on Computer Science and Information Technology, IEEE, 2008, pp. 923-929. [2] Huddar S R, Gowri S, Keerthana K, Vasanthi S, Rupanagudi S R. Novel algorithm for segmentation and automatic identification of pests on plants using image processing, International Conference on Computing Communication & Networking Technologies, IEEE, 2012, pp. 26-28. [3] Mallu B Deshishetti, M Y Teggi, Ambana Gowda Durgad. Growth and export performance of tomato in India, International Journal of Economic and Business Review, 2015, 3: pp. 48-52. [4] Devrim Unay, Bernard Gosselin, Olivier Kleynen, Vincent Leemans, Marie-France Destain, Olivier Debeir. Automatic grading of Bi- colored apples by multispectral machine vision, Computers and Electronics in Agriculture, 2011, 75: pp. 204-212. [5] Yousef Al Ohali. Computer vision based fruit grading system:design and implementation, Journal of King Saud University – Computer and Information Sciences, 2011, 23: pp. 29-36. [6] Nandi C S, Tudu B , Koley C, Machine Vision Based Techniques for Automatic Mango Fruit Sorting and Grading Based on Maturity Level and Size, Sensing Technology: Current Status and Future Trends II, 2013, 8: pp.27-46. [7] Blasco J , Aleixos N, Moltó E. Computer vision detection of peel defects in Citrus by means of a region oriented segmentation algorithm. Journal of Food Engineering, 2007, 81: pp. 535-543. [8] Bouganis A, Shanahan M. A vision-based intelligent system for packing 2-D irregular shapes, IEEE Transaction on Automation Science and engineering, 2007, 4: pp.382–394. [9] Garcia H C, Villalobos J R. Automated refinement of automated visual inspection algorithms, IEEE Transaction on Automation Science and engineering, 2009, 6: pp.514–524. [10] Bhandarkar S M, Luo X, Daniels R F, Tollner E W. Automated planning and optimization of lumber production using machine vision and computed tomography, IEEE Transaction on Automation Science and engineering, 2008, 5: pp.677–695. [11] Baltazar A, Aranda J I, Gonza lez-Aguilar G. Bayesian classification of ripening stages of tomato fruit using acoustic impact and colorimeter sensor data, Computers and Electronics in Agriculture , 2008, 60: pp.113–121. [12] Van Zeebroeck A, Van linden V, Darius P, De Ketelaere B, Ramon H, Tijskens E. The effect of fruit properties on the bruise susceptibility of tomatoes. Postharvest Biology and Technology, 2007, 45: pp.168–175. [13] Clement J, Novas1 N, Gazquez1 J A, Manzano-Agugliaro F. High speed intelligent classifier of tomatoes by colour, size and weight, Spanish Journal of Agricultural Research, 2012, 10: pp.314-325. [14] Dhanabal T, Debabrata Samanta. Computerized spoiled tomato detection, International Journal of Research in Engineering and Technology, 2013, 2: pp. 38-41. [15] Md. Rokunuzzaman, Jayasuriya H P W. Development of a low cost machine vision system for sorting of Tomatoes, Agricultural Engineering International: CIGR Journal, 2013, 15: pp.173-180. [16] Rebecca R Milczareka, Mikal E Saltveit, Casey Garveyc T, Michael J McCarthy. Assessment of tomato pericarp mechanical damage using multivariate analysis of magnetic resonance images, Postharvest Biology and Technology 2009, 52: pp.189–195. [17] Polder G, Van der Heijden G. Measuring ripening of tomatoes using imaging spectrometry, Hyperspectral Imaging for Food Quality Analysis and Control, Chapter 12, Academic Press / Elsevier, 2010, pp. 369-402. [18] Akinmusire O. Fungal species associated with the spoilage of some edible fruits in Maiduguri Northern Eastern Nigeria. Adv Environ Biol. 2011; 5:157–62. [19] Rawat S. Food spoilage: microorganisms and their prevention. Asian J Plant Sci Res. 2015;5(4):47–56. [20] Tournas VH, Katsoudas E. Mould and yeast flora in fresh berries, grapes and citrus fruits. Int J Food Microbiol. 2005;105(1):11–7. [21] Sindhi K, Pandya J, Vegad S. Quality evaluation of apple fruit: a survey. Int J Comput Appl. 2016;975:8887. [22] Shukla AK. Electron spin resonance in food science. Cam- bridge: Academic Press; 2016. [23] Bhargava A, Bansal A. Fruits and vegetables quality evaluation using computer vision: a review. J King Saud Univ-Comput Inf Sci. 2018. https://doi.org/10.1016/j.jksuci.2018.06.002. [24] Pandey R, Naik S, Marfatia R. Image processing and machine learning for automated fruit grading system: a technical review. Int J Comput Appl. 2013;81(16):29–39. [25] Cunha JB (2003) Application of image processing techniques in the characterisation of plant leafs. In: IEEE International Symposium on Industrial Electronics, pp 612–616 26. [26] Fu Y. Fruit freshness grading using deep learning (Master’s The- sis). New Zealand: Auckland University of Technology; 2020. [27] Koumpouros Y, Mahaman BD, Maliappis M, Passam HC, Side- ridis AB, Zorkadis V. Image processing for distance diagnosis in pest management. Comput Electron Agric. 2004;44(2):121–31. [28] Brosnan T, Sun DW. Inspection and grading of agricultural and food products by computer vision systems—a review. Comput Electron Agric. 2002;36(2–3):193–213. [29] Tewari VK, Arudra AK, Kumar SP, Pandey V, Chandel NS. Estimation of plant nitrogen content using digital image pro- cessing. Agric Eng Int CIGR J. 2013;15(2):78–86. [30] Ozyildiz E, Krahnstöver N, Sharma R. Adaptive texture and color segmentation for tracking moving objects. Pattern Recogn. 2002;35(10):2013–29. [31] Tripathi MK, Maktedar DD. A role of computer vision in fruits and vegetables among various horticulture products of agricul- ture fields: a survey. Inf Process Agric. 2019; 7:183. [32] Simonyan K, Zisserman A (2014) Very deep convolutional net- works for large-scale image recognition. arXiv:1409.1556 16. [33] Krizhevsky A, Sutskever I, Hinton GE. ImageNet classifica- tion with deep convolutional neural networks. Adv Neural Inf Process Syst. 2012; 25:1097–105. [34] Rafel C Gonzalez. Digital image processing, 3rd edition, Pearson Education India, 2009.

Copyright

Copyright © 2023 Aashi Singh Bhadouria. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.